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AN EXPONENTIAL AUTOREGRESSIVE-MOVING AVERAGE PROCESS EARMA (p , q ) : 
DEFINITION AND CORRELATIONAL PROPERTIES 



A. J. Lawrance 
University of Birmingham 
Birmingham, England 

and 

P. A. W. Lewis 
Naval Postgraduate School 
Monterey, California 

ABSTRACT 

A new model for pth-order autoregressive processes with 
exponential marginal distributions EAR(p) is developed and an 
earlier model for first order moving average exponential processes 
is extended to qth-order, giving an EMA(q) process. The correla- 
tion structure of both processes are obtained separately. A 
mixed process, EARMA(p,q), incorporating aspects of both EAR(p) 
and EMA(q) correlation structures is then developed. The 
EARMA(p,q) process is an analog of the standard ARMA(p,q) time 
series models for Gaussian processes and is generated from a 
single sequence of independent and identically distribution 
exponential variables. 
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INTRODUCTION 



The first-order autoregressive sequence, EAR(l), was 
introduced by Gaver and Lewis (1975-1978) with the primary aim 
of generalizing the Poisson model for point processes to one in 
which the intervals between events were correlated but still had, 
marginally, exponentially distributed intervals. The EAR ( 1 ) 
sequence is a simple random linear combination of independent 
exponential random variables whose properties are relatively 
simple to derive. This is in contrast to previous attempts to 
generalize the Poisson process via Markov dependence which led 
to intractable models (see e.g. Wold, 1948 and Cox, 1955) . 

In Lawrance and Lewis (1977) another sequence of dependent 
exponential random variables was introduced. This sequence, 
called EMA(l), was again a random linear combination of independent 
exponential random variables, but had the dependency properties 
of a first order moving average process . The first-order moving 
average and autoregressive processes were combined by Jacobs 
and Lewis (1977) to form the EARMA(1,1) sequence. Jacobs and 
Lewis (1977) gave stationary initial conditions and mixing 
properties of the sequences, these results applying to the EAR(l) 
and EMA(l) processes as special cases. 

In the present paper we extend these results and describe 
a mixed pth-order autoregressive, qth-order moving average process 
with exponential marginal distributions which we denote as 
EARMA (p, q) . The process is again a random linear combination 
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of independent exponential variables, and as such is simple to 

generate on a computer; it will thus be useful, for example, in 

simulation studies of queues with correlated interarrival times 

or service times (see Jacobs, 1978) . The process is not unique, 

but its correlation sequence {p } does satisfy equations like 

k 

the Yule-Walker equations which arise in the study of linear 
processes (see e.g. Feller, 1966, or Box and Jenkins 1970). 

It is perhaps well to reiterate the essential difference 
between the EARMA(p,q) process and the ARMA(p,q) process; this 
is that the EARMA(p,q) process is defined to have an exponential 
marginal distribution. It is not known how one would pick the 
error sequence in the ARMA(p,q) sequence to make it have, even 
approximately, marginal exponential distributions. In fact the 
marginal distributions would tend to be approximately normally 
distributed for most error sequences (Mallows, 1967); the catch 
in this result is that the distribution of the error sequence be 
independent of the parameters of the moving average and auto- 
regression. This is not so for the EARMA(p,q) process. 

This paper will be limited to definitions and to description 
of the correlational properties of the EARMA(p,q) process. In 
Section 2 the EAR(p) model is introduced and an explicit solution 
for the required error process for autoregression of order 2 is 
given. In Section 3 we describe the extensions of the EMA(l) 
model to the EMA(q) model; these are relatively straightforward 
and are indicated in Lawrance and Lewis (1977). The general 
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EARMA(p,q) model is introduced in Section 4, and specific results 
are obtained for the EARMA(1,1), EARMA(2,1) and EARMA(1,2) models in 
Section 5 . 

In deriving correlational properties it is assumed that 
the EARMA(p,q) process is stationary. The question of s tationarity , 
stationary initial conditions and mixing properties will be con- 
sidered elsewhere, as will be questions of distributions of sums 
of the dependent variables and spectra of point processes with 
EARMA(p,q) interval structure. There are also open questions of 
estimation of parameters and fitting to data. 

We note too that there is a mild degeneracy to the EAR(p) 
process in that one obtains runs in which the variables are 
scaled versions of the previous variables. This disappears when 
the moving average component is introduced . Another drawback is 
that, unlike the ARMA(p,q) model, only positive valued serial 
correlations can be obtained from the EARMA(p,q) model and while 
much data appears to be of this type (see e.g. Lewis and Shedler, 
1977), it is a drawback. This can be overcome by considering 
antithetic processes but this aspect of the model is beyond the 
scope of the present paper. 
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2. THE EXPONENTIAL AUTOREGRESSIVE EAR (p) MODEL 
2a . Definition . 

The standard linear, first-order autoregressive model 
for a stationary sequence of random variables {X^} is defined 
by the equation 

X. = pX. , + e. , i = 0,+l,+2, ... , (2.1) 

where p is a constant which is less than 1 in absolute value 
and {e^} is a sequence of independent and identically distributed 
random variables. Gaver and Lewis (1975-1978) showed that if 
the {X^} sequence were to have an exponential marginal distri- 
bution with parameter A, then the parameter p should be greater 
than or equal to zero and less than one, and should be zero 

with probability p and an exponential ( A ) random variable, E^, 
with probability 1-p. Thus 



X. = pX. . + £ . 
1 H 1-1 1 




i = 0 , +! , + 2 , . . . , 




px i-l 


w . p . 


CL 


_ 






i = 0 , +1 , +2 , . . . , (2.2) 




pX. , + E. 
M l-l l 


w.p . 


1-P, 



where {E^} is an i.i.d. sequence of exponential ( A ) random 
variables. Note that for this EAR(l) model the distribution of 
the depends on p, the multiplicative weight of X^_^. 
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This violates an assumption which is implicit in many applica- 
tions of (2.1), the so-called AR(1) model. In particular standard 
results showing that the { X ^ } sequence becomes a normal process 
as p -*■ 1 for any {e^} sequence are invalid; in the EAR(l) 
process the ' s always have, by construction, an exponential () ) 

marginal distribution. 

Generalization of the usual higher-order autoregressive 
models AR(p) based on extensions of (2.1) to higher order 
autoregressive exponential processes is difficult. This is 
because it is not possible to solve the defining equation for the 
distribution of the e^'s, if it exists. We present here a 
different type of p-th order autoregressive models with 
exponential marginal distributions. They share with the AR(p) 
models the same correlation structure, are p-th order Markov 
processes, and are (autoregressively ) functions of at least one 
of the previous p variables. 

The second-order model, EAR (2) , takes the form 



X . = 

l 



a l X i-l 



a»X . „ 

2 i-2 



w.p. 1-a, 



w . p , 



+ e . 

l 



(2.3) 



a. 



where a. and a„ are constants (0 < , a 2 < 1) and we show 



later that the distribution of the 



is uniquely determined 



by the requirement that the X_^ 1 s have exponential marginal 
distributions. The second-order autoregressive nature of the 



model is evident; X^ is always a function of one of 
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the previous two values X^_^ and X i-2 ’ T ^i s i- s in contrast 
to the AR(2) model in which X_^ is a function of a linear com- 
bination of X. , and X. 0 . 

l-l 1-2 

The third-order model is given by 



a l x i-l 


W.p. 


1 - a 2 






a 2 X i-2 


w.p. 


a 2 


+ £ i • 


(2.4) 


a 3 X i-3 


w.p. 


a 2 a 3 







The p-th order model is similarly constructed and may be 



written 



where 



f a l X i-l 

[ x 

2 i-2 



aJC. „ w.p. a 



w . p . a i \ 

2 



X. = 

l 



+ £ i ' 



(2.5) 



“ X. w.p. a 

P i-P P 



i 

a i = ■!, °j ' 

1 ^ 



l = 2 , . . . , p-1 



( 2 . 6 ) 



and 



a^ = (l-a 2 ) , 



a = n a . . 
P 1 = 2 3 



The mixing probabilities and the weights on the auto- 
regressed variables are to some extent a matter of choice (other 
parametrizations are clearly possible) and we have been guided 
by two considerations; having a minimum number of parameters, 
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preferably the same number of parameters as the order of the 
autoregression, and by the need for the autoregression in the EAR(p) 
model to reduce in order by one when the last coefficient, 't , 
is set to zero. In the present parametrization this implies the 
weak restriction that it is not possible to suppress intermediate 
autoregressions, i.e. dependence of X^ on X^_^ -*- n a EARMA(2) 

model . 

With regard to the question of parametrization, one could 
in (2.3) replace the probabilities a^= (l-a_) and a^ = by (1-p) 

and p and there is then no need for the weights and 

to be less than or equal to one. But if they are not, the process 
will not reach equilibrium unless p is suitably chosen, i.e. 
be stable. Again, even if the process is stable it is not clear 
yet that the additional parameter adds any generality to the 
process. We consequently consider only the parametrizations 
given in (2.6). 



2b. The error sequence 

We now obtain the distribution of the i.i.d. 

sequence which will ensure that the {X^} sequence 

model has an exponential marginal distribution. Let 

and 4> (s) be the Laplace-Stiel t jes transforms of 

i 

distributions of the X.'s and the s.'s: 

l l 



in the EAR ( 2 ) 

♦x. (s) 

1 

the marginal 



-X . s 

1 (s) = E(e 1 ) 

X . 



- . s 

(J) (s ) = E (e 1 ) . 

t. • 
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(2.7) 



Then from Equation (2.3) we have 

cf> (s) = [ (1-a ) ({> (a.s) + a _ 4> (a_s)] <(> (s) , (2.8) 

x 2 x ._ 1 1 2 X i _ 2 2 e 

l 

where we have used the fact that expectation of the mixture of 
two dependent random variables is the mixture of the expectations 
of the marginal random variables, here X^_^ and X^_ 2 . Thus 
we avoid the joint Laplace-Stietl jes transform which comes in 
when one tries to solve the usual linear AR(2) equations to 
obtain an exponential process. Assuming marginal stationarity 
for the process, we have 



4> x ( s ) = [(l-a 2 ) (^(o^s) + a 2 4> x (a 2 s)] <J> £ (s) . (2.9) 

To show that such an error sequence {e^} exists we solve (2.9) 

directly and invert the transform. We have for the EAR (2) model, 

using the key requirement that the marginal distribution of the 

X.'s be exponential ( A ) , and thus <l> v (s) = A/(A+s), that 
1 X 



<J> y (s) 

i / \ A 

S - (l-a 2 ) <J> (ot^s ) +~a^^Ja^s) 
(A+o^s) (A+a 2 s) 

(A+s) [ (l-a 2 ) (A+a 2 s) + a 2 ( A+a^s ) ] 
Then by a partial fraction expansion 



( 2 . 10 ) 



( 2 . 11 ) 
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®,: (S) = n 0 + "l JTs + "2 rSs ' 



where S = ( l+a^-a^ > a 2 ‘ Using the fact that and are 

probabilities, it is easily verified that 7 Tq , and tt^ are 

positive, and since their sum is equal to 1, we have 
0 <_ tTq, tt 1 , tt 2 <_ 1 • Thus e is a convex mixture of a discrete 
component and two exponentials, and thus has a proper distribution. 
This distribution is also unique, by the unicity theorem for 
Laplace-Stiel t jes transforms. The complete specification of 
£ i is, for i = 0, + 1, + 2, ... 



e . 

l 



0 



w 



.p. a^/{ l+a^-a 2 ) , 



E i w.p. (l-a 1 ) (l-a 2 )/[l-a 2 (l+a 1 -a 2 ) ] , 

E l /S w.p. (l-a 2 ) ( a;L -a 2 ) 2 /[ (l+a 1 -ra 2 ) (1-S) ] , 



( 2 . 12 , 



where {E^} is again an i.i.d. sequence of exponential ( X ) 
random variables. It is obvious from (2.12) that the mean and 
variance of depend on ct^ and oi 2 , Uhe multiplicative 

weights of and 2 respectively, as well as on X. 

As in the EAR ( 1 ) model there is a non-zero probability 
of e. being zero; otherwise it is E. or a scaled version of 
E^ . The higher order models can in principle be similarly 
treated, although above the third order there will be difficulty 
with the partial fraction expansion. 



q 



Correlation structure 



2c . 

The correlation structure of the stationary EAR(p) 
models can be obtained by the usual device of multiplying the 
defining equations for X^ by X^_ r , for r = 1,2,..., and 
taking expectations. What results are difference equations 
which are entirely analogous to the Yule-Walker type equations 
obtained for the standard AR(p) model. 

Thus taking the EAR(2) case as a typical example, we 
have from (2.3) that 

E(X i X i_r ) = (l-a 2 ) [a 1 E(X i _ 1 X i _ r ) +E(X i _ r ) E ( e ± ) ] 

+ a 2 [a 2 E(X i _ 2 X i _ r ) + E(X i _ r ) E ( e ± ) ] . (2.13) 

-1 -2 

Using the fact that E(X^) = A , var(X^) = A , because the 
process has an exponential marginal distribution, and from (2.3) 
that 

E(e) = (l-a 2 ) (l-a x +a ) E(X) , (2.14) 

we obtain for the correlations p r = Corr (X^ , X^_ r ) the equation, 
p r = a 1 (l - a 2 )p r _ 1 + a 2P r _2 ( r 2. D (2.15) 

with = p_ r and p^ = 1. For the general EAR(p) process 

there is the corresponding equation 
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(r > 1) • 



(2.16) 



p r _ a l a l p r-l + a 2 a 2 p r-2 + + a p a p p r-p 

Equation (2.15) is a system of second order difference 
equations from which we can obtain the following results. 

(i) The solution of the difference equation (2.15) is (see, 
for example. Box and Jenkins, 1970, pp . 58-59) 

p r = Y l z l + Y 2 Z 2 (r - 11 

,, 2. r ,, 2. r 

z 1 (l-z 2 )z 1 - z 2 (l-z 1 )z 2 

' (z x -z 2 ) (1+ Zl z 2 ) ' 

where z ^ and z 2 are reciprocals of the roots of the 
characteristic equation 

1 - a 1 (l-a 2 )B - a 2 B 2 = 0 , 

2 2 2 

and the roots are real since a^(l-a 2 ) + 4c* 2 >_ 0 . Also 
0 <_ z 2 < z^ < 1. An implication of these results is that 
the serial correlations are positive and eventually decay 
geometrically, i.e. like y^z^. We have assumed that a 2 > 0 
otherwise we have the EAR(l) model. 

(ii) The correlations and p 2 can be uniquely defined in 

terms of the parameters and a 2# and vice versa; this 

follows from (2.15) for r = 1 and r = 2. We have 



(2.17) 

(2.18) 
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(2.19) 



P 1 1 + a 



2 




and 



a 



2 




2\l/2 

P 2~ P 1 1 



a 



1 




1 



( 2 . 20 ) 



if a 2 ^ 0 . If a 2 = 0 the model reduces to the EAR ( 1 ) 



Equation (2.20) may be used to obtain Yule-Walker estimates 
of and a 2 from estimates of the first two serial 

correlations . 



that there are values of and a 2 for which p 2 > p^; 

thus the additional degree of autoregression produces, at 
least for the first two serial correlations, a broader 
correlation structure than is possible with the EAR(l) 
process (a 2 = 0 ) . 



(iv) One way to measure the amount of correlation which is intro- 



duced into the sequence { E-^ } by the autoregression is by 
an index of dispersion (Cox and Lewis, 1966, p. 71). This 
is just the limiting value of the variance of the sum of 
k adjacent to X^'s, standardized by its value for independent 
exponential variates : 



model of Gaver and Lewis (1975-1978), and p^ = a . 



(iii) If a 2 7 ^ 0 then, unlike the EAR(l) case in which p 2 = 

2 

we have p 2 > p^. This can be seen from (2.19), which can 
be written as P 2 = + a^d-pj) _> pj. Note too 
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J = lim 



var (X ) 



k { E (X)} 



k-1 

1 + 2 l 
j=l 



(1 - 



k )P j 



1 + 2 l P, . 
j = l 1 



( 2 . 21 ) 



and is proportional to the initial point of the spectrum 
of the process {X^}. For the EAR(2) process this is, 
using (2.17) , 



J 



1 + 



2Y 1 Z 1 

(l-z 1 ) 



+ 



2 y 2 z 2 

(l-z 2 ) 



( 2 . 22 ) 



This becomes very large as the roots and z 2 approach 

1, indicating that the process has very long term dependence 
in it . 



Some other properties of the EAR (2) process which are 



of 


interest are 


that the regression 


of X. on 

l 


X. , and 
l-l 


X. _ 
1-2 


is 


linear in the 


given values 


and x . _ 

i-2 


of X. , 
l-l 


and 



X 

i-2 



EIX.IX._i = X i-1 : 



X. _ 
1-2 



x . 

l- 



2 



) 



2 

(l-a„)a,x. . + a_x. « + A (l-a_) (1-a. +a_) 
2 1 l-l 2 1-2 2 12 



(2.23) 



and that the conditional correlations of X^ and X^_^, given 
X i — i , ... , are zero for £ = 3,4,... . 
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We note too that the EAR (2) model, like the EAR ( 1 ) model, 



is slightly degenerate in that one obtains runs of X^ ' s which 
are fixed multiples of the previous X^_^ or X f-2 ' 

Joint distributions, higher-order joint moments and partial 
sums of the {X^} process will be considered elsewhere. 



3. THE EXPONENTIAL MOVING AVERAGE EMA(q) MODEL 

The EAR(l) model of Gaver and Lewis (1975-78) led to the 
development by Lawrance and Lewis (1977) of a corresponding first- 
order exponential moving average model; this took the form, in 
the backward case, of 



X. 

l 



6E i w.p. 8 

(0<_8<1; i = 0 , +1 , + 2 , . . . ) (3.1) 

BEL + E i _ 1 w.p. (1-8 ) , 



where the {E^} is again a sequence of i.i.d. exponential ( A ) 

variables. The X^ ' s have an exponential marginal* distribution 

and are only serially dependent for lag one; this model is highly 

tractable and a full account of the statistically useful properties 

was obtained. The forward model is defined as a random mixture 

of BE. and BE. + E . . , instead of BE; and BE- + E - . 

l l it 1 i l l-l 

Lawrance and Lewis (1977) pointed out briefly that exten- 
sions to second-order moving-average models are possible; thus we 
replace E^_^ ; '' n (3.1) by another EMA(l) variable, a random 

linear combination of 8-.E. . and 8, E. . + E. which will still 

1 l-l l l-l i-2 

be exponentially distributed and independent of the E^ variable. 



Thus the second order backward model, EMA ( 2 ) , becomes 



3 2 E i 




w .p . 


3 2 




3 2 E i + 




w.p. 


(1-32)3, 


(3.2) 


3 2 E i + 


B l E i-l + E i -2 


w .p . 


(1-B 2 ) (1-3,) , 





where 0 <_ 3,, B 2 S. 1; i = 0,+l,+2, ... . The serial dependency 

of this model clearly stops at the second lag and 



P, - ( 1 — 3 2 ) B,U - (1-6 2 )B 1 >; P 2 = 6 2 (1 - 3,) (1 - B 2 ) . (3.3) 



This model reduces to the independence case and the EMA(l) model 
for various values of B, and 3 2 - T he general EMA(q) model takes 
the form 



3 E. 

q i 



x . = 

i 



q i 



w .p . b 



q +1 



1 i E * i 

q-1 l-l 




w.p . 


b 

q 


> i E • i 

q-1 l-l 


••• + B i E i- q+ i 


w.p . 


b 2 


+ 

i — 1 
1 

w 
1 — 1 
1 

tr 


+ 6 l E i-q+l + E i-q 


w.p. 


i — 1 

XI 



(3.4) 



for 0 £ 3,, 3 2 , ... , 3 <_ 1; i = 0, +1, +2, ... and 



i = q+1 , 



b. = < (1-3 ) (l-B.)B. , q > i > 2, 

Q 1 1-1 — — 



(1-3 ) ••• (1-3.) 
q ! 



i = 1 . 



(3.5) 
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Note that the 8^'s can be obtained uniquely from the b^'s; there 
are q+1 b^'s, but only q 3's, since the sum of the b^'s is equal 
to one. It is simple to see that the {X^} have exponential 
marginal distributions. The serial correlations for this model 
clearly have the cut-off property associated with moving average 
schemes; they can be obtained without recourse to difference 
equations. Premultiplication of (3.4) by X^_ r (r >_ 1) and then 
taking an expectation gives 



E (X . X . ) = B (b , + ••• + b, ) E (E . X . ) 

1 i-r q q+1 1 l l-r 

+ 6 , (b + • • • + b. ) E (E . , X . ) 

q-1 q 1 l-l i-r 



+ 6.. (b_ + b ) E(E. ,.X. ) + b E (E . X. ). (3.6) 

12 1 i-q+1 i-r 1 l-q i-r 



This simplifies since there are the relations 



6 i (b i + i + 



+ V = b i+ i 



(q > i > 1) 



(3.7) 



Thus on converting (3.6) to covariances we have 



Cov (X . , X . = Y b ,, Cov (E . ,X. 

l i-r L - q+l-m i-m' i-r 

m=0 M 



(3.8) 



The covariances on the right-hand side of (3.8) follow from (3.4) as 



) , , Var (E . ) 0 < m < q 

q-m+1 i-m — — M 



Cov (E . , X . ) = 

i-m i 



(3.9) 



otherwise . 
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Thus (3.8) becomes 




m=r 



with 



q-r+1 



v v+r 



(1 < r < q) 



P 



(q) _ 



Corr (X. ,X. 

1 l-r 



(3.11) 



r 



0 



(q+1 < r < °°) . 



Thus the serial correlations are just lagged products of the 
sequence and the formula (3.11) is completely analogous to the 
formula for the serial correlations of the standard MA(q) process; 
see Box and Jenkins, 1970, p. 68. It can be seen from (3.11) that 
all the correlations are nonnegative and it may further be shown 
that they are bounded above by 1/4. Note too that since the p^^ ' s 
are lagged products of the b^ sequence, it is not possible to 
determine the b^'s uniquely from the 's. Therefore it is 

not possible to uniquely determine the 6^'s from the 's. 



It can easily be shown, from (3.11), that for the EMA(q) process, 
this is given by 



This is maximized when all b s are equal and thus (?„ = l/(l + £) , 
l = 1,2, ...,q. These values of 3^ have the property that they 
give equal weights to the q+1 possible linear combinations which 



Consider now the index of dispersion J, defined at (2.21) . 



J = 2 




(3.12) 



can make up an X_^, that is b^ = l/(l+q), i = 1,2 
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/ • • • f 



q+1 . 



The maximum values of J are then, as q increases, 1.5, 1.666, 
1.750, 1.8000 and generally, 1 + q/(l+q) with 2 as limiting value; 
thus beyond a certain point, increasing the order of the moving 
average (which can conceptually go to infinity) , has little effect. 
This implies that the over all dependence in the process, as 
expressed by J, is bounded and that very high values of q do 
not substantially increase dependence. 

A convenient notation for the EMA(q) sequence {X^} is 
, meaning that has a moving average structure of order q 

over E^ , , E i_g using the parameters (3 , (3q_q/ . .., £3^. 

In this notation, can be expressed in terms of ^ by 

the recursion 



nf q) = 3 E. + 

l q l l l-l 



q 1,2,... 



(3.13) 



where { I 

l 

taking value 



is an independent sequence of binary variables 



zero with probability ^q+i = ^q * 



4. THE EXPONENTIAL AUTOREGRESSIVE-MOVING AVERAGE PROCESSES 
EARMA (p , q) . 

4a . Definitions 

We have defined both autoregressive processes and moving- 
average processes in exponential variables of any specified orders, 
p and q. Here we bring them together into a single process. 
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EARMA(p,q) , although it will be seen that the method of combination 
is not unique. We will then have a process of great flexibility 
in modelling dependent exponential variables, bearing favorable 
comparison with the standard ARMA(p,q) process in modelling dependent 
Gaussian variables. Jacobs and Lewis (1977) linked the two first 
order exponential processes EMA(l) and EAR(l), giving an EARMA(1,1) 
mixed model and obtained the serial correlations, some higher 
order explicit results and discussed central limit and mixing 
properties. For the general mixed process we shall give two types 
of model but restrict ourselves to their correlational properties, 
and in particular derive the difference equations satisfied by 
the serial correlations; these are similar but not identical to 
those of the standard ARflA process. The special process EARMA(2,1) 
and EARMA (1,2) will be considered in more detail. 

In seeking exponentially distributed mixed autoregressive- 
moving average processes we will work from the pure (backward) 

moving average process EMA(q) given in (3.4). One reason why 
the exponential moving average and exponential autoregressive 
models are appealing and tractable is that they are expressed 
in terms of independent exponential variables. If this property 
is to be carried over into the mixed models, then the autoregressive 
contribution should enter without violating this feature; thus, 
to construct the EARMA (p,l) process we replace the E^_^ variable 
in the EMA(l) of (3.1) by a|p|, an EAR(p) variable. This is 
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independent of E i in (3.1) because it is a function only of 

E. , £•_-,/ ••• • The defining equation for the EARMA (p, 1) process 
l™i. 1 ^ 

is thus 



X. 

i 



3E i w.p. 6 

( o £&<_ i ; i=o ,+l , +2 , . . . ) 

6 E i + a|p| w.p. 1-8 



(4.1) 



which is the model treated by Jacobs and Lewis (1977) when 

/ \ 

p = 1. Similarly, (3.4) leads, on replacing E i _ q by A^ 
to the EARMA (p,q) process, with equation 



X . = 

l 



8 E. 

q i 



w.p. b 



8 E. + 8 ,E. , 

q l q-1 l-l 



8 E. + 8 ,E. ,+•••+ 6 n E. . , 

q l q-1 1-1 1 l-q+l 



q+1 



w.p . b 



(4.2 



w.p . b 



8 E. +8 ,E. +* ,, + 3 n E. i + A.^^ w.p. b 

q l q-1 l-l 1 i-q+1 l-q ^ 



for i = 0 
0 <_ & , 
EARMA (p , q) 



+1, +2,..., where the b^'s are defined at (3.5) and 
, 8^ <_ 1 . Writing xfP ,c ^ as a variable in the 
process based on the moving average parameters 
.. , 8^f the mixed process can be defined recursively as 



8 E. 

q i 



w.p. 8 



x: 

i 



(p#q) _ 



8 E. + x. (p ' q-1) w.p. 1-6 
q l l-l ^ q 



( i =0 , +1 , +2 , . . . ) 



(4.3) 
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This class of models will sometimes be written as EARMA (p,a) 
to signify that it is based on a backward moving average. 

Consider further the structure of the mixed model; for instance, 



paralleling the standard model ARMA(p,q) model. In contrast 
to the standard model it is also possible that the autoregressive 
aspect could be absent in stretches of the process when one of 
the pure moving average selections is chosen each time. Dependency 
would still be retained in the model by the moving average part 
(apart from the q = 1 case of course) ; while this is not at 
all unnatural there can be other situations when it is desired 
to always have autoregressive dependency. Such considerations 
lead to alternative mixed models; initial concern at the non- 
uniqueness of these models is best allayed by realizing that 
there is nothing unique about the standard Gaussian mixed 
ARMA (p , q ) models. In an alternative formulation of the general 
mixed model, to be denoted by EARMA + (p,q) , the shifted form of 
the forward moving average, briefly mentioned in Section 3 is 
used. The retention of independence between successive terms 
in the model then leads to the EARMA + (p,l) process as 



X^ depends on E^, E 



and not on E . , , , E „ 
l+l i+2 



I 




w.p. 



X . 



i 




(i = 0 , +1 , +2 , ...) 



( 4 . 4 ) 
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and to the EARMA + (p,q) as 



e a<p> 

q i-q 



w -p* Vi' 



3 A. (p) + 3 i E . - 

q l-q q-1 i-q+1 



w.p. b g/ 



X. = 

l 



(4.5) 



6 A. (p) +6 -i E . , +• • •+ 3-.E. 

q i-q q-1 i-q+1 1 1-1 



w.p. b 9 , 



,3 A. (p) + 3 n E. ,, +•••+ 3-.E. -I + E. 

' q l-q q-1 i-q+1 1 l-l i 



w.p. b^ , 



for i = 0, +1, +2, ... . It can be seen from (4.5) that this 

has the structure 



x (p#q) 

i 



f 

(■ 



6 A. ( P> 

q i-q 



B A<P> + X. (<3 ‘ 1) 
q i-q i 



w.p. 



3 



q 



w.p. 1-3 



( i-0 , +1 , +2 , . . .) 



(4.6) 



where is a variable in the shifted forward moving average 

model of order q-1 using the parameters 3 qf ••• / 3q • Thus 

the autoregressive dependence is always present, and is lagged 
q values in arrears; the moving average variable gives greater 
flexibility to the initial form of the dependence, and differs for 
the two models. In EARMA (p,q) the most recent E^ is always 
included; then with probability (1-3^) 3^_q a linear combination 
of Eq and E q_q i s included, and so on moving back; thus 
because of the certain addition of a new E. each time there 

l 

cannot be runs of scaled values; further, this is a back progression, 
natural in many cases. The price of these features is that the 
model can exhibit patches of independence when only the E^ is 
chosen, i.e. the autoregressive tail is not chosen. This cannot 
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happen in the EARMA + (p,q) where the autoregressive dependency 
is always present; however, complicated but weak runs of scaled 
values are just possible in the EARMA + (p,q) model, arising from 
a low order autoregressive contribution sucessively being chosen 
on its own. Such a situation would be extremely rare. Our general 
feeling is that in practice there would not be much to choose 
from between the two types; a third type, more similar to 
EARMA + (p,q) than the other can be formed by interchanging the 
processes in (4.6) with a suitable shifting of scale. This is 
not considered here. 



4b . Correlations for the backward mixed model EARMA (p,q) 

We next derive equations satisfied by the serial 
correlations of the EARMA - (p , q) process, denoted here simply as 
EARMA (p , q) • Multiplying each side of the defining equations 
(4.2) by r (r ^ 0) and taking expectations, gives 



E(x i x i . r ) = 6 q (b q+1 + ••• +■ bp E(E iXi . r ) r = +1, 12, .. 

+ Vl% + ••• + b l> E(E i-l X i-r ) 

+ ••• + h (b 2 + b l> E<E i-q+l X i-r ) 



+ b,E(A. (p) X. ) 

1 i-q l-r 



(4.7) 



This equation is not valid for r = 0 since the expectation of 
the mixture is not the mixture of the expectations when the 
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variables are identical. Following equations (3.6), (3.7) and 

(3.8), the covariance form of (4.7) becomes 



Cov(X.,X i _ r ) 



q-1 . > 

= 7 b Cov (E . ,X. ) + b, Cov (A . P , X . ) 

L r . q+l-m l-m l-r 1 l-q' l-r 

m=0 ^ ^ 



(4.8) 



It now becomes easier to work mainly in terms of correlations and 
to define 



p = Cov (X. ,X. ) 

r i' l-r 



and 



K = Corr (E. ,X. ) 

r i' l-r 



(4.9) 



for r = 0, +1, +2, ... . Since the and X^ variables have 

the same marginal exponential distribution, (4.8) becomes 



q ; 1 

- I b 

m=0 



. , K + b, Corr (A. (p) , X. ) 
q+l-m r-m 1 i - q i-r 



(4.10) 



for r = +1, +2, ... . To calculate the cross-covariances 

between the autoregressive and mixed process, we first note 



that 



.. (p) = 

i-q 



a A (p J 

1 i-1-q 

a,A<e> 

2 i-2-q 



a A< 0 > 

p i-p-q 



w .p . 



w.p . 



w.p . 




i = 0 , +1 , +2 , ... 

(4.11) 



24 



following the notation at (2.5) and (2.6). The 



e term in (4.11) 



has a distribution which depends only on E^ , such as was determined 
for p = 2 in Section 2. Multiplication of (4.11) by x ^_ r in 
order to calculate correlations leads to 



We now wish to substitute from (4.10) for the correlations in (4.12) 
and so obtain a difference equation for p . However we do not 
have (4.10) in the case r = 0. Thus in (4.12) when i-r-(i-i-q) =q 
this substitution is not possible, that is when SL = r if 
r <_ p. In this case 




l a.a p Corr(A 
1=1 




) + Cov (e . ,X. ) /Var (E) 

l-q l-r 



r 



0 , + 1 , +2 



(4.12) 



Corr (A. (p) ,X. ) = b, 
l-q l 1 



(4.13) 



as may be seen from (4.2). Thus (4.10) and (4.12) lead to 



q-1 



P 



r 





+ b 1 Cov(e. ,X. )/Var(E) 
1 i-q i-r 



(4.14) 
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for r 



is omitted. 



2 

= l,2,...,p; if r > p then the term a r a r ^i 
The asterisk denotes that the Z = r term is omitted from this 
summation when r <_ p . The equation simplifies on defining 

p„ = and C = Cov(e. , X. )/Var(E) ; (4.15 

0 1 r-q i-q' l-r ' ' 

equation (4.14) may then be written 
P q v 1 

p r _ a £, a £ P r-t + ^q+l-m^ K r-m 

1=1 m =0 ^ 

This is the desired general result. 

Noting that = 0 for j > 1, we see that for r > p+q, 

(4.16) reduces to an rth order difference equation 



?* 

I « 
1=1 



„ } + b,C . (4.16 

r-£-m 1 r-q 



p r = a l a l p r-l + “2 a 2 p r-2 + 



+ a a p 

P P r-p 



(r > p+q) , 



(4.17 



which is the same as (2.16) for the EAR(p) process. To calculate 

the initial p+q-1 serial correlations, Di t • ...o . , ( we need 

^1 p+q-l 

Kq , K , ... , and Cq, ... , C _ q+ 1 ^ explicit 

expressions for these quantities are given in the Appendix. The 
correlation structure of EARMA(p,q) processes is thus similar to 
that of the standard ARMA(p,q) processes; the only difference is 
that initial calculation of the first p+q serial correlations 
are needed to start the difference equation (4.17), rather than 
the first p as in the ARMA(p,q) case. Note finally that (4.16) 
is only strictly true when p 1 , q >_ 1 , although similarities 
with the equations for the (p = 0, q) case and (p, q = 0) case 
are apparent. 
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These results all apply to the backward model, EARMA ( p , q ) ; 
for the forward EARMA + (p,q) model (see (4.5)) slightly 
different correlation equations apply. The main difference is 
that (4.8) is now 



Corr (X. ,X. ) = 

1 l-r 



b . n Corr (A. (p) ,X. ) 

q+1 l-q i-r 



m=l 



b , K ^ . (4.18) 

q+l-m r-q+m 



Using (4.12) we then find, corresponding to (4.16), that 



P q P* 

p = T a.a.p „ + T b , , {K , - ) b n , ) + b 

r l l r-i q+l-m r-q+m r-H-q+m 



q+l ( r-q 



(4.19; 



for r = 1,2, 



of K Q , K_ 1 , . 



with 



b^ + ^; as before explicit calculation 



' K — p-q+ 2 and c 0' c -r 



, C ,, are 
-q + 1 



required to obtain the first p+q serial conditions. Further 
comparisons between the two types of model will be dealt with 
elsewhere . 



5. SPECIAL CASES OF THE EARMA (p,q) PROCESS 

We shall give the explicit versions of the correlation 
equations (4.16) in the cases (p = 1, q = 1) , (p = 2, q = 1) 
and (p = 1, q = 2); these are considered to be potentially the 
most useful. 
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(i) EARMA (1,1) 



This model will be written in the notation 



X. = 
1 



w.p. 6 



BE^ + w.p. 1-B 



where 



A. = pA . . + e . , 

l l-l l 



e . = 

l 



w.p, 



E^ w.p. 1-p 



The correlation equations, from (4.16), are then 



P 1 = a i a i b i + b 2 K l + b i C 0 ' p r = a l a l p r-l ' (r 2. 2) 



giving 



P-L = P (1-B) + (1-p) 6(1-B), P r = PP r _]_, 



(r > 2) 



This agrees with the result (2.4) of Jacobs and Lewis (1977). 



(ii) EARMA (2,1) 

Using (4.16) to obtain the results for p^ and p^ we have 
2 

P 1 = a l a l b l + a 2 a 2 P -l + b 2^ K l " K -l^ + b l C 0 '* 

Thus 
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(l-a 2 a 2 ) p 1 



= a ± a 1 (1-6) 2 - a 1 a 1 (3 (1-3) ( tt 1 -t- t^S -1 ) + (tt 1 + S 1 ) ( 1-6 ) , 



and 



P 2 = 



a i a i P l + 



a 2 a 2 



with 



P 



r 



a l a l p r-l + 



a 2 a 2 p r-2 



(r > 3) . 



(iii) EARMA (1,2) 

In this case (4.16) gives 

2 

P 1 = a i a i b i + b 3 K i + b 2 K o + b l C -l ; 

p 2 = a l a i p l + b 3 (K 2 _K l^ + b 2^ K l _K 0 ) + b l C 0' 

and 

p = a., a, p . (r>3). 

r 11 r-1 — 

These simplify to 

p ± = p (1-3 1 ) 2 (1 - B 2 ) 2 + 3 1 6 2 (1-S 2 ) + (tt 1 + S _1 ti 2 ) B 1 (1-B 1 ) ( 1 -15 2 ) 2 , 
P 2 - pp x - 6 1 B 2 (l-6 2 ) + (ir 1 + S _1 tt 2 ) ( 1-B^ ) B 2 (l-B 2 ) , 

and 

p r = a i a i p r _i (r >_ 3) . 
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6 . FURTHER DEVELOPMENTS 

There are many facets and properties of the EARMA(p,q) 
process which will be investigated in later papers. Some of these 
properties have been investigated for the EAR(l) process, the 
EMA(l) process and the EARMA(1,1) process by Gaver and Lewis 
(1975-78), Lawrance and Lewis (1977) and Jacobs and Lewis (1977) 
respectively. They include mixing conditions, infinite divisibility, 
stationary initial conditions, joint distributions, distributions 
of sums, spectra and higher order correlations. It should also 
be emphasized that all the serial correlations in the EARMA(p,q) 
process are positive; this aspect of the process can be broadened 
by considering antithetic processes and will be discussed else- 
where . 

Another important question which arises is whether there 
are non-normal distributions other than the exponential for which 
mixed autoregressive, moving average structures can be defined 
analogous to EARMA(p,q) . The general question is difficult and 
has been considered by Gaver and Lewis (1975-78) for the first 
order-autoregressive process. However it is clear that by adding 
two independent EARMA (p , q) processes, say and { X 5 2 ^ } , 

we obtain a process which has Gamma (2) marginal distributions, 



P{X. < X. (1) + xf 2) < x} 

l—i i — 



2Ax 

/ v e V dv 

0 
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and ARMA(p,q) correlation structure. Some analysis shows that 
this process is generated directly as a mixture over two independent 
i.i.d. exponential ( X ) sequences, {e|^} and {e|^}, possibly 
scaled. It would be interesting to extend this process to the 
fractional Gamma case, i.e. < not an integer but it is not 
clear whether this process exists or how to construct it. 

For the first-order autoregressive case, it is simple to 
show that the random variable defined at (2.12) is infinitely 

divisible and therefore that the solution of (2.4) for exists 

when is Gamma ( k,A) and has a Laplace-Stiel jes transform 

which is just (2.11) raised to the power k , 

E (e S£;l ) - (p + (1-p) , k > 0, A > 0 (6.1) 

However it is difficult to invert this transform, or to generate 
the random variable on a computer unless < is an integer. 

This should give an indication of some of the interesting 
theoretical questions which are raised by the EARMA process. 
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APPENDIX: Calculations of {K_^_, r = 0,1,2, . ..} and 

{C_ r , r = ,l,...,q-l} for EARMA(p,q) models 



By definition 



K = Corr(E.,X. ) = Corr(E. ,X.) 
-r 1 l+r l-r 1 



(A.i: 



By the usual process of multiplication and expectation it is found 
that 



Cov (E . ,X.) = 



• f A . / \ 

l-r 1 i 



b L , Var(E. ) 0 < r < q-1 , 

q-r+1 l-r — — ^ 

b, Cov(E. , a{ P ^) q < r < » , 

1 l-r l-q — 



(A. 2) 



and in terms of correlations 



q-r+1 



0 < r < q-1 , 



K 



-r 



(A. 3) 



b, Corr (E. ,A. (p) ) q < r < 

1 l-r i-q — 



The calculation of the cross-correlations between the independent 
exponential sequence and the derived autoregressive sequence 
proceeds in the usual way, and gives 



Corr (E . , A ) = Cov (E . , e . ) /Var (E) = d_ ; 
i i i l ' 0 



(A. 4) 



Corr (E . . , A f ) 

i-D i 



min ( j ,p) 

l 

1=1 



a i a i CorrIEi 



(a. 5 ; 
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We only need (A. 5) when P >. 2 and for j = l,2,...,p-2. 
Recursively then (A. 4) and (A. 5) give 



Corr (E . .,A. (p) 

i-D i 




(j = l) 




( j = 2) 




(j=3) (A. 6) 





Further expressions will be evident based on the fact that the 
sura of the suffices equals j, that all possible such groups 
of terms are present, and that the coefficient of a particular 
term is the number of distinct orders of a term of that type. 
By definition we have 



C_ r = Cov(e i ,X i+r )/Var (X) = Cov (e^ , X i ) /Var (X) . 



(A. 7) 



As at (A. 2) above we have 




0 < r < q-1 . 



(A. 8) 



Hence 
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C = b , d A 
-r q+l-r 0 



(A. 9 : 



The calculation of (A. 8) depends on the form of the error random 
variable e. as a function of the random variable E.. For 

l l 

example, in the EAR (2) case. 



d 0 = *1 + V s • 



(A. 10) 
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